{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Seaborn\n",
    "\n",
    "那么Pandas与Seaborn之间有什么区别呢？\n",
    "\n",
    "其实两者都是使用了matplotlib来作图，但是有非常不同的设计差异\n",
    "\n",
    "1. 在只需要简单地作图时直接用Pandas，但要想做出更加吸引人，更丰富的图就可以使用Seaborn\n",
    "2. Pandas的作图函数并没有太多的参数来调整图形，所以你必须要深入了解matplotlib\n",
    "3. Seaborn的作图函数中提供了大量的参数来调整图形，所以并不需要太深入了解matplotlib\n",
    "4. Seaborn的API：https://stanford.edu/~mwaskom/software/seaborn/api.html#style-frontend"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 泰坦尼克号数据分析\n",
    "\n",
    "这是是历史中著名的海难事件，大量游客在事故中丧生，也有部分游客获救。现在这里有一份数据给出一批乘客的信息如姓名、年龄、性别、票价等等一些信息，和是否获救，然后让你建模分析，再去预测另一批乘客的获救与否。我们一起来看看\n",
    "\n",
    "## 掌握数据概况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline\n",
    "sns.set()\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\", category=DeprecationWarning)\n",
    "from sklearn.utils.testing import ignore_warnings\n",
    "\n",
    "def warn(*args, **kwargs):\n",
    "    pass\n",
    "import warnings\n",
    "warnings.warn = warn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = pd.read_csv(\"titanic/train.csv\")\n",
    "df_test = pd.read_csv(\"titanic/test.csv\") # 留作练习让你们分析"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- PassengerId => 乘客ID\n",
    "-　Survived => 是否获救\n",
    "- Pclass => 乘客等级(1/2/3等舱位)\n",
    "- Name => 乘客姓名\n",
    "- Sex => 性别\n",
    "- Age => 年龄\n",
    "- SibSp => 堂兄弟/妹个数\n",
    "- Parch => 父母与小孩个数\n",
    "- Ticket => 船票信息\n",
    "- Fare => 票价\n",
    "- Cabin => 客舱\n",
    "- Embarked => 登船港口"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "粗略观察一下数据，发现age里有不少缺失，Cabin（舱号）大量缺失，其他属性个别缺失"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(9,5))\n",
    "sns.heatmap(df_train.isnull(), cbar=False, cmap=\"YlGnBu_r\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 这些是类别列\n",
    "cols = ['Survived', 'Sex', 'Pclass', 'SibSp', 'Parch', 'Embarked']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nr_rows = 2\n",
    "nr_cols = 3\n",
    "\n",
    "fig, axs = plt.subplots(nr_rows, nr_cols, figsize=(nr_cols*3.5,nr_rows*3))\n",
    "\n",
    "for r in range(0,nr_rows):\n",
    "    for c in range(0,nr_cols):  \n",
    "        \n",
    "        i = r*nr_cols+c       \n",
    "        ax = axs[r][c]\n",
    "        sns.countplot(df_train[cols[i]], hue=df_train[\"Survived\"], ax=ax)\n",
    "        ax.set_title(cols[i])\n",
    "        ax.legend() \n",
    "        \n",
    "plt.tight_layout()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 认识数据\n",
    "\n",
    "- 第一张图：？\n",
    "- 第二张图：？\n",
    "- 第三张图：？\n",
    "- 第四，五张图：？\n",
    "- 第六张图: ？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 看看年龄的因素 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bins = np.arange(0, 80, 5)\n",
    "g = sns.FacetGrid(df_train, row='Sex', col='Pclass', hue='Survived', margin_titles=True, size=3, aspect=1.1)\n",
    "g.map(sns.distplot, 'Age', kde=False, bins=bins, hist_kws=dict(alpha=0.6))\n",
    "g.add_legend()  \n",
    "plt.show()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 分析一下"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 看看你票价因素 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train['Fare'].max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bins = np.arange(0, 550, 20)\n",
    "g = sns.FacetGrid(df_train, row='Sex', col='Pclass', hue='Survived', margin_titles=True, size=3, aspect=1.1)\n",
    "g.map(sns.distplot, 'Fare', kde=False, bins=bins, hist_kws=dict(alpha=0.6))\n",
    "g.add_legend()  \n",
    "plt.show()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 仓位因素"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.barplot(x='Pclass', y='Survived', data=df_train)\n",
    "plt.ylabel(\"Survival Rate\")\n",
    "plt.title(\"Survival as function of Pclass\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.barplot(x='Sex', y='Survived', hue='Pclass', data=df_train)\n",
    "plt.ylabel(\"Survival Rate\")\n",
    "plt.title(\"Survival as function of Pclass and Sex\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 登船口因素"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.barplot(x='Embarked', y='Survived', data=df_train)\n",
    "plt.ylabel(\"Survival Rate\")\n",
    "plt.title(\"Survival as function of Embarked Port\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.boxplot(x='Embarked', y='Fare', data=df_train)\n",
    "plt.title(\"Fare distribution as function of Embarked Port\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 增加一些新维度\n",
    "\n",
    "### 家庭大小，单独，名字长度，称呼"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for df in [df_train, df_test] :\n",
    "    \n",
    "    df['FamilySize'] = df['SibSp'] + df['Parch']\n",
    "    \n",
    "    df['Alone']=0\n",
    "    df.loc[(df.FamilySize==0),'Alone'] = 1\n",
    "    \n",
    "    df['NameLen'] = df.Name.apply(lambda x : len(x)) \n",
    "    df['NameLenBin']=np.nan\n",
    "    for i in range(20,0,-1):\n",
    "        df.loc[ df['NameLen'] <= i*5, 'NameLenBin'] = i\n",
    "    \n",
    "    \n",
    "    df['Title']=0\n",
    "    df['Title']=df.Name.str.extract(r'([A-Za-z]+)\\.') #lets extract the Salutations\n",
    "    df['Title'].replace(['Mlle','Mme','Ms','Dr','Major','Lady','Countess','Jonkheer','Col','Rev','Capt','Sir','Don'],\n",
    "                    ['Miss','Miss','Miss','Mr','Mr','Mrs','Mrs','Other','Other','Other','Mr','Mr','Mr'],inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.subplots(figsize=(10,6))\n",
    "sns.barplot(x='NameLenBin' , y='Survived' , data = df_train)\n",
    "plt.ylabel(\"Survival Rate\")\n",
    "plt.title(\"Survival as function of NameLenBin\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 结论？？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = sns.factorplot(x=\"NameLenBin\", y=\"Survived\", col=\"Sex\", data=df_train, kind=\"bar\", size=5, aspect=1.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 结论？？\n",
    "\n",
    "### 称呼因素"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.subplots(figsize=(10,6))\n",
    "sns.barplot(x='Title' , y='Survived' , data = df_train)\n",
    "plt.ylabel(\"Survival Rate\")\n",
    "plt.title(\"Survival as function of Title\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.crosstab(df_train.FamilySize,df_train.Survived).apply(lambda r: r/r.sum(), axis=1).style.background_gradient(cmap='summer_r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 结论？？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据清洗\n",
    "\n",
    "### 第一步填充缺失数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 根据称呼补充他们的性别\n",
    "df_train['Title'] = df_train['Title'].fillna(df_train['Title'].mode().iloc[0])\n",
    "\n",
    "# 年龄使用平均值填充\n",
    "df_train.loc[(df.Age.isnull())&(df.Title=='Mr'),'Age']= df_train.Age[df.Title==\"Mr\"].mean()\n",
    "df_train.loc[(df.Age.isnull())&(df.Title=='Mrs'),'Age']= df_train.Age[df.Title==\"Mrs\"].mean()\n",
    "df_train.loc[(df.Age.isnull())&(df.Title=='Master'),'Age']= df_train.Age[df.Title==\"Master\"].mean()\n",
    "df_train.loc[(df.Age.isnull())&(df.Title=='Miss'),'Age']= df_train.Age[df.Title==\"Miss\"].mean()\n",
    "df_train.loc[(df.Age.isnull())&(df.Title=='Other'),'Age']= df_train.Age[df.Title==\"Other\"].mean()\n",
    "df_train = df_train.drop('Name', axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 设置登船口默认值是第一个\n",
    "df_train['Embarked'] = df_train['Embarked'].fillna(df_train['Embarked'].mode().iloc[0])\n",
    "# 票价用平均值填充\n",
    "df_train['Fare'] = df_train['Fare'].fillna(df_train['Fare'].mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 年龄按10年分段，票价按50分段，方便查找规律\n",
    "df = df_train\n",
    "df['Age_bin']=np.nan\n",
    "for i in range(8,0,-1):\n",
    "    df.loc[ df['Age'] <= i*10, 'Age_bin'] = i\n",
    "\n",
    "df['Fare_bin']=np.nan\n",
    "for i in range(12,0,-1):\n",
    "    df.loc[ df['Fare'] <= i*50, 'Fare_bin'] = i        \n",
    "\n",
    "# 把文字变成数字，让计算机更好处理\n",
    "df['Title'] = df['Title'].map( {'Other':0, 'Mr': 1, 'Master':2, 'Miss': 3, 'Mrs': 4 } )\n",
    "# 如果称呼为空，填充第一个\n",
    "df['Title'] = df['Title'].fillna(df['Title'].mode().iloc[0])\n",
    "df['Title'] = df['Title'].astype(int) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 复制一份数据，保护原始数据\n",
    "df_train_ml = df_train.copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train_ml.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 把类别参数做成新的列，用０－１表示对应项\n",
    "df_train_ml = pd.get_dummies(df_train_ml, columns=['Sex', 'Embarked', 'Pclass'], drop_first=True)\n",
    "df_train_ml.drop(['PassengerId','Name','Ticket', 'Cabin', 'Age', 'Fare_bin'],axis=1,inplace=True)\n",
    "df_train_ml.dropna(inplace=True)\n",
    "df_train_ml.drop(['NameLen'], axis=1, inplace=True)\n",
    "df_train_ml.drop(['SibSp'], axis=1, inplace=True)\n",
    "df_train_ml.drop(['Parch'], axis=1, inplace=True)\n",
    "df_train_ml.drop(['Alone'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train_ml.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 下一步就是机器学习，有兴趣同学可以看 sklearn\n",
    "\n",
    "官网：https://scikit-learn.org/stable/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
